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READMISSIONS:  METHODOLOGY  AND  MEANING* 


W.  DUKE  MORTON,  M.S.**,  Research  Associate 
ALMA  LANTZ,  M.A.,  Student  Assistant 
JOSEPH  HALPERN,  PH.D.,  Research  Consultant 
Program  Information  and  Analysis  Department 
Fort  Logan  Mental  Health  Center,  Denver,  Colorado 


When  visitors  touring  a hospital  ask  about  its  readmission  rate,  they 
seldom  realize  the  complexity  of  the  question.  And  when  they  are  shown 
the  various  ways  in  which  readmissions  can  be  modeled,  each  resulting  in 
a different  picture  of  hospital  functioning,  they  are  likely  to  say,  “That’s 
all  very  interesting,  but  what  is  the  real  readmission  rate?’’ 

There  is  no  one  “real  readmission  rate.”  Numbers,  no  matter  how 
they  are  presented  statistically,  are  neutral;  they  carry  no  inherent  value 
judgments.  But  a careful  examination  of  the  various  models  or  meanings 
that  can  be  given  to  readmission  figures  is  a first  step  toward  deriving 
value  judgments  that  are  appropriate  to  the  situation  being  evaluated. 

Tradition  assumes  that  a patient’s  readmission  to  a hospital 
signifies  a failure  to  cure,  and  hence,  that  a high  readmission  rate  must 
indicate  a poor  treatment  program.  The  use  of  readmission  figures  as  a 
criterion  of  program  effectiveness  embodies  so  many  conceptual  and 
methodological  problems  that  one  is  tempted  to  say  that  the  whole  sub- 
ject would  be  best  forgotten.  However,  such  a statement  would  deny  the 
value  of  readmission  figures  as  a measure  of  what  is  happening  in  a 
dynamic  system  that  treats  patients. 

On  the  assumption  that  readmissions  may  be  a valid  criterion  of 


*This  study  was  based  on  data  supplied  by  the  Fort  Logan  Record 
System  Project.  The  Record  System  was  developed  with  the  assistance  of 
Public  Health  Service  Grant  No.  5-R01-MH-14872-08  from  the  National 
Institute  of  Mental  Health. 

**3520  West  Oxford  Avenue,  Denver,  Colorado  80236. 
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program  effectiveness,  we  studied  four  aspects  of  the  topic:  (a)  readmis- 
sion rates;  (b)  readmissions  and  the  concept  of  envelopment;  (c)  re- 
admissions, length  of  hospitalization,  and  time  in  the  community  prior  to 
readmission;  and  (d)  readmissions  as  a function  of  personality  charac- 
teristics. Although  each  area  was  studied  in  reference  to  the  treatment 
program  at  the  Fort  Logan  Mental  Health  Center,  the  results  are  dis- 
cussed in  terms  of  evaluative  criteria  applicable  to  any  mental  health 
facility. 


BASIC  METHOD  FOR  AREAS  STUDIED 

In  the  broadest  sense,  anyone  who  has  been  discharged  from  a given 
mental  health  facility  and  seeks  subsequent  treatment  for  the  same  dis- 
order could  be  considered  a readmission  to  mental  health  care.  For  the 
purposes  of  this  study,  a readmission  was  defined  as  a patient  admitted 
to  and  discharged  from  Fort  Logan’s  Adult  Psychiatry  Division,  with  a 
subsequent  admission  to  the  same  facility  and  the  same  division. 

Population  Pool 

To  insure  that  the  data  contained  only  individuals  who  had  had  an 
opportunity  to  be  readmitted,  patients  (N  = 488)  who  had  not  been  dis- 
charged from  their  first  admission  as  of  June  30,  1967  (the  end  of  the 
1966-67  fiscal  year),  were  excluded.  Known  dead  (N  = 74)  were  excluded 
unless  a readmission  had  occurred  prior  to  death.  Patients  (N  = 61)  who 
sought  care  in  the  Adult  Psychiatry  Division  following  discharge  from 
another  division,  or  vice  versa,  and  those  (N  = 92)  originally  admitted  to 
the  Adult  Psychiatry  Division  and  transferred  to  another  division,  or 
vice  versa,  were  also  eliminated  from  the  study  sample. 

The  remaining  population  consisted  of  2,198  patients  who  had  been 
admitted  and  discharged  prior  to  June  30,  1967  and  subsequently  read- 
mitted or  not  readmitted  to  the  Adult  Psychiatry  Division  by  December  31, 
1967.  Our  initial  work  had  indicated  that  six  months  would  allow  ample 
time  for  readmission  of  the  majority  of  patients  who  were  discharged  just 
prior  to  June  30  and  who  would  seek  subsequent  care.  Later  examination 
of  the  data  indicated  that  the  six-month  period  should  have  been  extended 
to  a full  year. 
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Unit  of  Analysis 

The  original  admission  cohort  was  the  unit  of  analysis.  The  word 
“cohort”  designates  a group  of  patients  who  have  a certain  characteristic 
in  common-in  this  case,  first  admission  in  a given  fiscal  year.  If  a 
patient  was  originally  admitted  in  the  1961-62  fiscal  year,  he  remained  in 
that  sample  no  matter  when  he  was  readmitted.  Table  1 shows  the  result- 
ing data  pool. 


TABLE  1 

READMISSIONS  AND  NON-READMISSIONS  BY 
ORIGINAL  ADMISSION  YEAR* 


Original  Admission  Year 

Readmissions 

Non-re  admissions 

Tot  al 

1961-62 

51 

103 

154 

1962-63 

147 

313 

460 

1963-64 

137 

354 

491 

1964-65 

112 

298 

410 

1965-66 

77 

290 

367 

1966-67 

60 

256 

316 

Totals 

584 

1614 

2198 

* Excluding  known  dead,  multidivision  users,  interdivision  transfers, 
and  patients  net  discharged  from  their  first  admission  as  of  June  30,  1967. 


The  original  admission  cohort  was  used  to  determine  whether  there 
were  any  common  characteristics  among  the  patients  admitted  in  a given 
year  who  sought  subsequent  care  following  discharge.  Such  an  analysis 
also  permitted  a year-by-year  replication  of  the  study  and  examination  of 
any  trends  that  might  emerge  across  the  years. 

READMISSION  RATES 

Some  may  view  these  readmission  figures  as  statistical  tricks  that 
somehow  hide  or  modify  the  true  meaning  and  facts  about  readmissions. 
No  tricks  were  used.  In  the  context  of  this  paper,  the  meaning  of  readmis- 
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Fig.  1.  Distribution  of  readmissions  over  fiscal  years. 
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sion  figures  cannot  be  separated  from  the  facts.  For  example,  three 
methods  for  modeling  readmission  figures  are  discussed.  The  conceptual 
advantages  and  disadvantages  of  each  model  are  presented,  with  an 
explanation  of  possible  statistical  artifacts  inherent  in  each  model. 

The  three  models  of  calculation  described  in  this  section  reveal 
different  facts  and  have  different  meanings.  The  usefulness  of  each 
depends  upon  the  question  asked.  These  three  methods  by  no  means 
exhaust  the  ways  of  obtaining  readmission  figures;  however,  thev  are 
commonly  used  and  illustrate  the  various  advantages  and  disadvantages 
in  determining  a readmission  rate. 

Ratio  ol  Readmissions  to  Total  Admissions 

One  of  the  most  popular  ways  of  calculating  readmission  rate 
relates  the  number  of  readmissions  in  a given  year  to  the  total  admis- 
sions during  that  time.  This  method  offers  the  hospital  administration  an 
ecological  measure  of  readmissions  that  has  some  statistical  disadvant- 
ages but  offers  a good  index  for  planning  future  services.  The  measure 
indicates  the  degree  to  which  a given  year  is  saturated  with  readmissions, 
and  using  this  calculation,  Fort  Logan’s  readmission  rate  increases  from 
2%  to  30%  during  the  first  six  years  of  operation  (see  Fig.  1). 

One  disadvantage  to  this  procedure  is  that  relating  the  number  of 
readmissions  to  total  admissions  predisposes  the  readmission  rate  to 
increase.  The  readmission  rate  for  the  1961-62  fiscal  year  must  be  low 
because  it  was  the  first  year  of  the  hospital’s  operation  and  few  patients 
had  had  time  to  be  discharged.  By  1964-65,  a large  number  of  patients 
had  been  discharged;  thus,  more  patients  had  been  readmitted  and  more 
patients  were  “at  risk”  of  being  readmitted.  Hypothetically,  after  twenty 
years  of  operation,  Fort  Logan  could  have  treated  and  discharged  so 
many  patients  that  the  patient  load  would  consist  entirely  of  readmitted 
patients.  At  the  same  time,  however,  the  Center  could  be  maintaining 
75%  of  its  discharged  patients  in  the  community. 

A second  disadvantage  is  that  such  a calculation  obscures  the  role 
of  multiple  readmissions  or  readmission  events  in  the  increasing  re- 
admission rate.  Almost  40%  of  the  Fort  Logan  patients  who  are  admitted 
for  the  second  time  (first  readmission)  come  back  a third,  fourth,  fifth,  or 
sixth  time.  Table  2 shows  that  over  30%  of  the  later  admission  years’ 


FIRST  READMISSIONS  AND  (MULTIPLE  READMISSIONS) 
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YEAR  OF  READMISSION 

1967-68 

(9)  I 

4 (8) 

7 (9) 

13  (8) 

31  (4) 

56  (37) 

39.8% 

1966-67 

2 (3) 

11  (17) 

17  (16) 

33  (18) 

41  (10) 

29  (4) 

133  (68) 

33.8% 

1965-66 

2 (2) 

20  (29) 

31  (15) 

42  (20) 

23  (1) 

118  (67) 

36.2% 

1964-65 

10  (5) 

29  (18) 

50  (11) 

30  (4) 

119  (38) 

24.2% 

1963-64 

12  (6) 

56  (9) 



35  (3) 

103  (18) 

14.9% 

1962-63 

22 

30  (2) 

52  (2) 

3.7% 

1961-62 

co 

CO 

0% 

N umber  of 
Readmission 
Events 

69 

228 

190 

163 

96 

68 

Total  readmission  load 
for  each  admission 
year 

Percent  of  multiple 
readmits  in  the  total 
readmission  sample  from 
each  admission  year 

Original 
Admission 
Y ear 

1961-62 

1962-63 

1963-64 

1964-65 

1965-66 

1966-67 

Excluding  multidivision  users,  and  interdivision  transfers. 
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readmission  load  consisted  of  multiple  readmissions.  This  figure  also 
appeared  to  be  increasing  because  there  had  been  more  time  for  patients 
to  have  multiple  readmissions. 

Ratio  of  New  Patients  to  Subsequent  Readmissions  of  Same  Patients 

The  second  method  of  calculating  readmission  rate  compares  the 
total  number  of  new  patients  admitted  in  a fiscal  year  with  the  number  of 
these  patients  who  are  subsequently  readmitted.  Calculated  in  this 
manner,  the  readmission  rate  at  Fort  Logan  has  declined  from  33%  to  19% 
over  the  first  six  years  of  operation  (see  Fig.  1 and  Table  2).  This 
measure  offers  administration  a means  of  projecting  the  number  of  new 
admissions  each  year  who  will  eventually  return. 

This  system,  too,  has  its  disadvantages.  First,  the  decline  in  the 
readmission  rate  is  a statistical  artifact  produced  by  the  passage  of 
calendar  time.  For  example,  the  1961-62  original  admission  cohort  had 
had  six  years  in  which  to  be  discharged  and  readmitted.  Thus,  few 
patients  in  the  non-readmitted  group  were  likely  to  be  readmitted  at  a 
future  date.  However,  the  1966-67  original  admission  cohort  had  had,  at 
most,  a year  and  a half  in  which  to  be  readmitted.  Thus,  that  non-re- 
admitted group  contained  many  patients  likely  to  be  readmitted  at  a later 
date. 

Second,  this  calculation  does  not  consider  the  role  of  multiple 
readmissions.  Since  each  readmitted  patient  was  counted  only  once,  the 
proportion  of  patients  who  had  multiple  readmissions  cannot  be  deter- 
mined. It  is  interesting  to  note  that  both  methods  of  calculating  the 
readmission  rate  play  down  the  role  of  multiple  readmissions.  Yet  it  is 
likely  that  future  studies  will  show  that  the  multiple  readmission  has  a 
tremendous  influence  on  the  hospital’s  ability  to  offer  services  and  the 
treatment  teams’  ability  to  care  for  their  patients. 

Cumulative  Percentages  of  Readmissions 

Because  calendar  time  appears  to  be  the  major  problem  in  determin- 
ing readmission  rates  that  do  not  contain  statistical  artifacts,  a method 
was  used  that  holds  time  constant.  The  first  year  after  discharge  was 
divided  into  quarterly  units  and  all  subsequent  time  periods  into  years, 
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*Figures  exclude  known  deaths,  multiple  division  users  and  inter-division  transfers.  It  should  be  noted  that 
the  last  box  in  each  row,  which  represents  the  last  year  in  which  patients  could  be  readmitted,  actually  repre- 
sents only  half  of  the  year  since  the  cut-off  date  for  a readmission  was  December  31,  1967,  and  the  1967-68 
fiscal  year  extends  until  June  30,  1968. 

**PA  - Population  at  risk  as  of  December  31,  1967. 
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and  then  the  cumulative  percentage  was  determined  for  each  original 
admission  cohort  (the  population  “at  risk”)  readmitted  in  each  time 
period.  The  results  of  this  analysis  are  presented  in  Table  3.  Except  for 
the  1965-66  fiscal  year,  the  cumulative  readmission  rates  are  amazingly 
stable  and  suggest  that  the  readmission  rate  at  Fort  Logan  has  neither 
increased  nor  decreased  appreciably  since  its  first  year  of  operation. 

Using  the  same  method,  Table  4 addresses  the  question;  “What 
percentage  of  the  readmitted  patients  were  readmitted  within  the  first  90 
days  following  discharge,  and  so  on,  until  December  31,  1967,  the  cut- 
off date?”  Again,  except  for  the  1965-66  fiscal  year,  the  figures  are 
remarkably  stable.  Since  only  about  50%  of  the  patients  were  readmitted 
within  six  months  of  discharge,  it  might  have  been  more  appropriate  to 
extend  the  cut-off  date  another  six  months  to  obtain  a larger  sample  of 
readmitted  patients.  However,  the  year-by-year  stability  of  the  data 
indicated  that  this  technique  for  holding  time  constant  may  be  valuable 
in  future  studies. 

Although  the  third  method  of  calculating  the  readmission  rate 
avoids  the  difficulties  of  statistical  artifacts  induced  by  the  passage  of 
calendar  time,  it,  like  the  preceding  methods,  considers  a readmission  as 
a single  event  and  thus  makes  it  difficult  to  consider  the  role  of  multiple 
readmissions. 

Theoretically,  none  of  the  figures  produced  by  these  three  methods 
of  calculation  is  more  “correct”  than  the  others.  However,  by  correctly 
applying  each  of  the  models  to  the  questions  they  are  best  suited  to 
answer,  the  hospital’s  administrators  would  be  less  likely  to  make  in- 
appropriate conclusions  concerning  its  readmission  figures. 

READMISSIONS  AND  THE  CONCEPT  OF  ENVELOPMENT 

To  avoid  some  of  the  inherent  difficulties  of  readmissions  as 
evaluative  criteria,  McPartland  and  Richart  (1)  suggested  a scheme  of 
classifying  readmitted  patients  instead  of  reporting  all  readmissions  as 
a homogeneous  group.  Their  concept  of  “envelopment”  was  based  on  the 
varying  amounts  of  time  and  hospital  resources  patients  consumed. 
Based  on  the  length  of  time  to  readmission  and  the  degree  of  envelopment 
to  which  the  patient  returned,  readmissions  were  classified  as  follows: 

Delayed  transfer.  Any  patient  returning  to  a “less”  enveloping 


TIME  PERIODS  IN  WHICH 
READMITTED  PATIENTS  WERE  READMITTED 
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modality,  irrespective  of  the  time,  and  thus  considered  to  have  made  some 
progress. 

Unsuccessful  discharge.  Any  patient  readmitted  within  two  calen- 
dar months  of  his  previous  discharge. 

Relapse.  A patient  discharged  from  a service  and  later  readmitted 
to  the  same  or  a more  enveloping  service. 

If  a patient  returned  to  a more  enveloping  modality,  he  was  assumed  to 
be  more  impaired.  Similar  logic  was  used  for  the  “same”  and  “less” 
categories.  Envelopment  provides  two  ways  of  determining  types  of 
readmissions. 

Discharge-to-admission  Method  of  Comparison 

The  first  method  compares  the  previous  discharge  treatment 
modality  with  the  current  admission  modality.  It  answers  the  question, 
“How  much  more  enveloped,  or  impaired,  are  readmitted  patients  at  the 
time  of  their  second  admission  than  at  their  time  of  discharge?” 

Using  this  method,  Rutledge  and  Binner  (2)  studied  all  Fort  Logan 
admissions  between  July  1962  and  June  1963  who  were  readmitted 
(including  multiple  readmissions)  by  December  1965.  They  adapted  the 
concept  of  envelopment  by  ranking  Fort  Logan  services  as  follows: 

Rank  Service 

1 Inpatient 

2 Family  Care 

3 Day  Care 

4 Halfway  House 

5 Night  Care 

6 Home  Care 

7 Outpatient 

Only  15.22%  of  the  total  sample  could  be  classified  as  “Delayed 
Transfers,”  i.e.,  having  returned  to  a less  enveloping  modality.  In  the 
categories  of  “Unsuccessful  Discharges”  and  “Relapses,”  28.26% 
returned  to  the  same  modality  and  56.51%  to  more  enveloping  modalities. 
Thus,  over  half  of  the  patients  readmitted  were  assumed  to  have  been 
more  impaired  upon  readmission  than  upon  previous  discharge.  (See 
Table  5 for  details  of  distribution.) 

In  updating  the  Rutledge-Binner  study,  we  made  two  changes: 


DISTRIBUTION  OF  KINDS  OF  READMISSIONS  BY  TIME  FROM  DISCHARGE  DATE 
(Admissions  between  July  1961  and  June  1963  readmitted  by  December  1965) 
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(a)  we  exluded  multiple  readmissions,  and  (b)  we  used  a revised  ranking 
that  gave  the  Halfway  House  treatment  modality  a rank  of  3 and  Day  Care 
a rank  of  4.  The  ranking  was  revised  in  the  initial  phase  of  a study  in 
progress  at  the  Center  on  the  chronic  patient  and  comprehensive  com- 
munity mental  health  centers. 

Our  analysis  (Fig.  2)  showed  a trend  over  the  years  toward  more 
unsuccessful  discharges  and  relapses,  with  fewer  delayed  transfers. 

Table  6 compares  our  findings  with  those  of  Rutledge  and  Binner. 
There  was  a drop  of  7%  in  delayed  transfers,  an  increase  of  3%  in 
unsuccessful  discharges,  and  an  increase  of  4 % in  relapses. 

TABLE  6 

COMPARISON  OF  CURRENT  RE  ADMISSIONS 
TO  RUTLEDGE-BINNER  DATA 


Kind  of  Readmission 

Rutledge-Binner  Data 

Current  Data 

Delayed  Transfer 

15.22% 

8.01% 

Unsuccessful  Discharge 

19.13% 

22.52% 

Relapse 

65.65% 

69.46% 

Although  these  differences  may  be  the  result  of  the  slightly  differ- 
ent samples  and  ranking  procedures,  in  both  studies  two-thirds  of  the 
readmissions  fell  into  the  relapse  category. 

Admission-to-admission  Method  of  Comparison 

The  second  method  compares  the  previous  admission  treatment 
modality  with  the  current  admission  modality.  It  answers  the  question, 
“How  much  more  enveloped,  or  impaired,  are  readmitted  patients  at  the 
time  of  their  second  admission  than  at  the  time  of  their  first  admission?” 
When  admission-to-admission  modalities  were  compared,  only 
23.8%  of  the  patients  returned  to  a more  enveloping  modality  and  hence 
were  assumed  to  be  more  impaired  upon  readmission  than  upon  first 
admission. 
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Methodological  Problems 

Each  comparison,  in  answering  the  question  for  which  it  was 
designed,  yielded  different  results.  In  monitoring  programs,  hospital 
administrators  must  weigh  the  answers  against  the  limitations  of  the 
model.  The  concept  of  envelopment  involves  the  tenuous  assumption  that 
degree  of  envelopment  is  a valid  indicator  of  impairment.  Circumstances 
may  influence  the  modality  to  which  the  patient  is  admitted  as  well  as 
the  modality  from  which  he  is  discharged.  Does  the  patient  have  a place 
to  live?  Does  the  team  have  space  for  the  patient?  The  use  of  just  two 
data  points,  discharge  and  readmission  or  admission  and  readmission, 
fails  to  consider  the  number  of  modality  movements  or  changes  a patient 
makes  during  hospitalization  and  may  not  adequately  reflect  the  role  of 
admission  circumstances. 

Both  methods  of  determining  kinds  of  readmissions  produce  arti- 
facts that  could  distort  the  picture  of  Fort  Logan’s  readmission  popula- 
tion. The  majority  of  patients  enter  day  care  or  in-hospital  care,  but  many 
are  discharged  from  outpatient  care,  the  least  enveloping  service.  The 
latter  patients  can  return  only  to  the  same  or  a more  enveloping  modality. 
Thus,  the  comparison  of  discharge-to-admission  modality  is  likely  to 
indicate  a high  percentage  of  patients  who  are  more  enveloped  (impaired) 
at  the  time  of  their  second  admission. 

The  admission-to-admission  comparison  has  the  same  methodologi- 
cal problem  in  the  opposite  direction.  Since  the  customary  modality  on 
first  admission  is  day-  or  in-hospital,  there  is  little  chance  to  go  to  a 
more  enveloping  modality  upon  readmission. 

Perhaps  one  answer  to  these  methodological  difficulties  would  be 
to  examine  a patient’s  “envelopment  career,”  the  total  degree  of  en- 
velopment during  a first  hospitalization  compared  with  that  for  subsequent 
hospitalizations. 

Although  both  methods  of  comparison  are  logical  possibilities 
within  the  Fort  Logan  program,  we  made  a year-by-year  comparison 
between  the  readmitted  and  non-readmitted  patients  to  determine  whether 
both  possibilities  were  warranted. 

The  discharge-to-admission  method  appears  to  be  valid  because  of 
the  increasing  tendency  to  discharge  subsequently  readmitted  patients 
from  the  more  enveloping  modalities  (Table  7).  If  envelopment  is  used 
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as  an  index  of  impairment,  readmitted  patients  are  more  impaired  at 
discharge  than  non-readmitted  patients  and  are,  therefore,  more  likely  to 
return.  The  steady  increase  in  discharges  from  24-hour  care  and  day  care 
suggest  that  the  teams  may  not  be  taking  the  time  to  bring  the  patient 
back  into  the  community  gradually  by  using  outpatient  care  for  a number 
of  weeks  prior  to  discharge. 

It  appears  that  the  logical  considerations  for  using  an  admission-to- 
admission  analysis  may  not  be  warranted.  Table  8 indicates  a slight 
trend  toward  more  enveloping  first -admission  modalities  for  patients  who 
return.  Collapsed  over  years,  47.3%  of  the  patients  who  returned  had  been 
admitted  originally  to  24-hour  care,  whereas  41.6%  of  the  non-returning 
patients  had  been  admitted  to  24-hour  care.  There  are  also  strong  in- 
dications that,  in  general,  Fort  Logan  has  been  admitting  fewer  patients 
to  day  care  and  more  to  in-hospital  care. 


LENGTH  OF  HOSPITALIZATION  AND  TIME  IN  COMMUNITY 
PRIOR  TO  READMISSION 

A previous  report  (3)  indicated  that  the  concept  of  length  of  stay 
can  also  be  modeled  in  a variety  of  ways.  To  simplify  matters  for  the 
purposes  of  our  concern  with  readmissions,  length  of  hospitalization  was 
defined  as  the  total  number  of  days  the  patient  was  considered  under 
care. 

A point-biserial  correlation  between  length  of  stay  and  readmission/ 
non-readmission  showed  no  relationship  (r  = -.013).  Since  patients  current 
as  of  June  30,  1967  were  removed  from  the  study,  the  data  were  biased 
toward  short  stays  on  the  part  of  the  non-readmitted  group.  Therefore, 
only  the  first  three  years’  data  were  analyzed.  The  resulting  correlation 
was  r = -.16  (p  = .05),  indicating  the  shorter  the  stay  the  more  likely  a 
patient  is  to  return.  This  is  a weak,  though  reliable,  relationship,  and  the 
use  of  a small  sample  statistic,  the  point-biserial  correlation,  on  a large 
sample  may  have  produced  the  result. 

It  has  often  been  assumed  that  short  periods  of  hospitalization 
involve  a necessary  “trade-off”  with  more  readmissions,  but  the  correla- 
tion obtained  in  this  study  indicates  that  this  may  not  be  true.  A correla- 
tion of  -.16  accounts  for  only  3%  of  the  variance. 
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To  further  explore  the  relationship  between  length  of  stay  and  re- 
admissions,  a scatter  plot  between  length  of  hospitalization  and  time  in 
the  community  prior  to  readmission  was  developed.  The  results  revealed 
that  (a)  the  longer  a patient  was  hospitalized,  the  shorter  the  stay  in  the 
community,  and  (b)  the  shorter  the  patient’s  hospitalization,  the  longer 
the  time  in  the  community  prior  to  rehospitalization.  Some  of  the  missing 
variance  between  length  of  stay  and  readmission  could  be  accounted  for 
by  the  long-stayers  who  can  maintain  only  short  periods  of  time  in  the 
community  before  they  must  return.  Why  this  relationship  exists  is  an 
important  question.  Is  the  treatment  program  somehow  training  patients 
to  be  successful  patients  instead  of  successful  community  members? 

Although  the  relationship  between  length  of  hospitalization  and 
readmissions  needs  more  study,  our  analysis  strongly  suggests  that 
looking  at  only  one  variable,  such  as  length  of  stay,  is  inadequate.  No 
one  variable,  analyzed  in  isolation,  can  provide  the  needed  information 
or  account  for  a maximum  amount  of  variance.  Instead,  relationships 
between  variables  must  be  considered.  The  examination  of  such  relation- 
ships may  be  complex,  involved,  and  difficult  to  interpret,  but  the 
questions  mental  health  professionals  are  asking  about  readmissions  are 
exactly  that:  complex,  involved,  and  difficult  to  analyze. 


READMISSIONS  AS  A FUNCTION  OF 
PERSONALITY  CHARACTERISTICS 

In  this  phase  of  the  study,  we  attempted  to  determine  the  personality 
characteristics  that  differentiated  the  non-read mitted  from  the  readmitted 
patient.  Information  on  each  patient  was  obtained  from  the  hospital 
admission,  mental  status,  and  social  history  forms. 

Analysis  was  based  on  a series  of  frequencies  and  probabilities 
computed  for  each  fiscal  year  for  each  of  the  categories,  readmissions 
and  non-readmissions.  The  expected  probabilities  of  readmission  were 
as  follows:  1961-62  33% 

1962- 63  32% 

1963- 64  29% 

1964- 65  27% 

1965- 66  21% 

1966- 67  19% 
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The  probability  that  a given  patient  was  a readmission  or  non- 
readmission as  a function  of  the  responses  to  the  items  on  the  forms  was 
then  obtained.  On  any  given  item,  the  number  of  responses  in  each 
category  was  divided  in  accordance  with  the  proportion  of  readmitted 
respondents  to  non-readmitted. 

If  the  observed  probability  differed  markedly  (by  5%  if  25%  of  the 
readmission  sample  was  included,  or  by  8%  if  20%  of  the  sample  was 
included)  from  the  expected  probability,  the  item  was  considered  to 
discriminate  between  readmissions  and  non-readmissions,  and  an  inference 
was  drawn. 

The  number  of  readmissions  for  the  later  years  is  low,  because 
there  has  not  been  sufficient  time  for  a readmission  to  occur.  Thus,  there 
are  a number  of  false  non-readmitted  patients  in  the  later  years’  com- 
parisons. We  expected  that  the  existence  of  these  false  non-readmissions 
would  reduce  the  number  of  discriminations  between  the  readmitted  and 
non-readmitted  populations  in  the  later  years;  however,  this  effect  oc- 
curred only  with  the  mental  status  form  and,  even  then,  the  trends  in  the 
data  were  surprisingly  consistent  with  the  discriminations  found  in  the 
in  the  earlier  years. 

Admission  Form  Data 

The  admission  form  characterized  the  readmitted  patient  as  single, 
having  experienced  difficulty  holding  a job,  having  been  under  some  form 
of  psychiatric  care  at  the  time  of  admission,  and  as  having  been  admitted 
to  24-hour  care  more  than  expected. 

Social  History  Form  Data 

The  data  showed  that  readmitted  patients  experienced  overprotec- 
tion, inconsistency,  dominance,  and  psychological  trauma  during  develop- 
mental periods.  There  were  also  indications  that  the  readmitted  patients 
come  from  broken  homes,  have  difficulty  with  the  law,  have  low  social 
productivity,  see  their  problems  as  primarily  centered  around  their 
families,  vocations,  and  financial  matters;  have  a history  of  mental  dis- 
turbance in  the  family;  and  exhibit  depression,  disorganization,  aggres- 
sion, and  antisocial  behavior  as  their  primary  symptoms. 
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Mental  Status  Form  Data 

The  strongest  findings  from  the  mental  status  examination  con- 
cerned cognitive  ability.  While  intelligence  estimates  ranged  from  high  to 
low,  the  readmitted  patients  exhibited  delusional,  phobic,  and  depressive 
thought  content;  blocking  and  slowing  of  thought  processes;  poor  or- 
ganization of  thought;  poor  use  of  logic; and  little  capacity  for  abstraction. 
Readmitted  patients  also  used  inappropriate  gesturing;  were  suspicious; 
moved  predominantly  against  people;  and  had  a poor  self-image,  poor 
prior  adjustment,  and  a gradual  rather  than  sudden  onset  of  illness. 
Oddly  enough,  they  had  few  emotional  disturbances  in  comparison  to  the 
non-read mitted  patients.  This  finding  was  especially  strong  for  the  first 
admission  cohort. 

Summing  the  information  from  all  three  forms,  the  readmitted 
patient  appeared  to  be  a marginal  individual  who  has  difficulty  com- 
municating with  others  effectively.  The  consistency  with  which  the  items 
mentioned  discriminated  from  year  to  year  indicated  that  treatment  teams 
may  be  unable  to  change  many  characteristics  of  the  readmitted  patients. 
Group  therapy  often  emphasizes  the  emotional  aspects  of  a person’s 
problems,  but  the  major  difficulties  of  readmitted  patients  centered 
around  inability  to  stabilize  in  the  community  and  inability  to  communi- 
cate with  others.  Such  patients  may  return  because  they  are  unable  to 
communicate  the  extent  of  their  impairment. 

Since  the  data  also  indicated  that  readmitted  patients  are  likely  to 
come  from  broken  homes  (where  mental  disturbance  has  a long  history), 
have  difficulty  with  the  law,  and  show  low  social  productivity,  perhaps 
treatment  should  focus  on  helping  the  patient  adapt  to  his  environment. 


SUMMARY 

While  it  may  appear  that  we  have  so  complicated  readmissions  as 
to  make  them  useless  as  criteria  for  program  evaluation,  actually  we 
have  done  the  opposite.  Our  study  demonstrated  that  each  of  the  many 
ways  in  which  readmission  figures  can  be  presented  is  designed  to 
answer  a specific  question.  The  various  methods  of  calculating  and 
describing  readmissions  do  not  represent  statistical  tricks  for  hiding 
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foreboding  truths  about  hospital  functioning.  Instead,  they  determine  the 
extent  to  which  readmission  figures  can  answer  a given  question  and, 
thus,  the  extent  to  which  those  figures,  in  combination  with  other 
measures,  can  gauge  the  effectiveness  of  a hospital  treatment  program. 
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The  Children’s  Division  of  the  Fort  Logan  Mental  Health  Center 
offers  24-hour  care  and  day  care  to  emotionally  disturbed  children  between 
the  ages  of  six  and  fifteen  who  require  controls  and  protection  that  cannot 
be  provided  in  their  own  homes  or  through  other  community  resources. 
Outpatient  services  are  available  only  to  children  who  have  graduated 
from  the  24-hour  or  day  care  programs.  The  treatment  program  provides 
milieu  therapy  for  all  children  and  gives  both  children  and  parents  in- 
dividual therapy,  group  therapy,  and  family  unit  therapy  as  needed. 

Admission  is  voluntary;  families  may  apply  directly  or  through 
referral  from  an  agency  in  the  community.  This  policy  has  called  for 
continuous  work  with  mental  health  agencies  and  juvenile  courts,  which 
have  been  accustomed  to  using  court  commitments  to  send  patients  to 
state  hospitals.  The  control  of  patient  intake  frees  the  program  staff  to 
use  all  of  its  clinical  manpower  for  treatment,  but  at  the  same  time,  it 
imposes  the  responsibility  of  insuring  that  children  who  could  benefit 
only  from  this  type  of  program  are  not  denied  admission. 

Each  application  for  admission  is  reviewed  by  the  staff  to  deter- 
mine the  best  course  of  action.  If  the  initial  information  from  the  family 
or  from  the  referring  agency  appears  to  meet  admission  requirements,  a 
diagnostic  evaluation  is  completed  at  the  community  level.  The  evalua- 
tion procedure  includes  interviews  with  the  child  and  with  his  parents, 
review  of  background  information  from  other  agencies,  and  consultation 
with  the  staff  of  the  referring  agency.  Children  are  not  accepted  for 
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admission  if  (a)  the  primary  diagnosis  is  that  of  mental  retardation  or 
brain  damage,  unless  there  are  emotional  problems  not  directly  related 
to  these  conditions,  (b)  there  is  physical  illness  severe  enough  to  require 
pediatric  care  in  a general  hospital,  or  (c)  antisocial  behavior  is  ex- 
hibited without  evidence  of  anxiety,  depression,  or  internalized  conflict. 
If  admission  to  the  Children’s  Division  is  deemed  inappropriate,  the  staff 
recommends  alternatives— return  to  the  referring  agency  or  other  possible 
sources  of  help. 

To  measure  the  effectiveness  of  our  preadmission  evaluation  pro- 
gram, a study  was  made  of  children  referred  to  Fort  Logan  during  1966 
and  the  first  quarter  of  1967  who  were  not  accepted  for  evaluation  or  who 
were  evaluated  but  not  accepted  for  admission.  The  purpose  of  the  study 
was  to  determine  (a)  the  adequacy  of  decisions  made  by  Fort  Logan  staff 
relevant  to  admission,  and  (b)  the  appropriateness  of  referrals  to  other 
agencies. 


METHOD 

The  study  group  consisted  of  50  children  and  their  parents  who 
contacted  the  Division  but  were  not  clinically  evaluated  and  42  children 
and  their  parents  who  were  evaluated  by  Fort  Logan  staff. 

Background  information  was  obtained  from  the  Center’s  intake 
forms,  historical  and  diagnostic  materials  from  referring  agencies,  sum- 
maries of  telephone  contacts  with  families  who  were  not  evaluated,  and 
reports  of  clinical  evaluations  by  Fort  Logan  staff.  Information  on  the 
current  adjustment  of  the  92  children  was  gathered  from  telephone  inter- 
views of  parents  or  reports  from  other  agencies.  Parents  of  77  of  the 
the  children  were  reached  by  telephone,  and  all  cooperated  in  supplying 
the  information  requested.  The  status  of  the  other  15  families  was  ob- 
tained through  the  agencies  to  whom  the  families  had  been  referred. 

RESULTS 

Outcome  of  Referrals  for  Families  Not  Evaluated 

Table  1 shows  the  referral  distribution  by  reported  benefit  or  lack 
of  benefit  for  this  group.  Information  was  obtained  on  the  four  families 
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for  whom  referral  was  deemed  inappropriate  as  well  as  on  the  46  families 
for  whom  other  agency  services  were  recommended. 

TABLE  1 

REFERRAL  DISTRIBUTION  OF  FAMILIES  NOT  EVALUATED 

(N  - 50) 


Disposition 

Outc 

Satisfactory 

ome 

Unsatisfactory 

Total 

Followed  through  with  referral 

To  original  agency 

13 

i 

14 

To  different  agency 

11 

6 

17 

Did  not  follow  through  with 

referral 

11 

4 

15 

Not  referred 

3 

1 

4 

Totals 

38 

12 

50 

Of  the  31  families  who  followed  through  with  referral,  24  reported 
that  the  agency  to  which  Fort  Logan  referred  them  had  been  of  benefit. 
Seven  families  considered  the  referral  outcome  unfavorable.  Two  of  these 
had  had  clinical  services  but  claimed  that  no  benefit  was  derived;  two 
families  stated  that  the  waiting  list  had  been  too  long;  three  children  had 
been  refused  service  because  family  income  was  too  high  or  because  the 
agency’s  pattern  of  services  did  not  meet  the  child’s  needs.  One  of  the 
three  eventually  received  private  care  with  good  results. 

On  the  basis  of  information  received  from  the  15  families  who  did 
not  follow  through  with  referral,  11  of  the  children  seemed  to  be  function- 
ing well.  Ten  of  the  11  had  manifested  spontaneous  improvement,  and  the 
11th  child  had  eventually  received  outpatient  psychiatric  care  to  which  he 
responded  favorably.  Four  of  the  15  children  were  not  faring  well.  The 
family  of  one  had  a negative  attitude  toward  any  kind  of  treatment,  the 
family  of  another  was  still  indecisive  about  getting  help,  and  the  family 
of  a third  was  seeking  psychiatric  care  for  a severely  mentally  retarded 
child.  The  fourth  child  received  care  in  a private  hospital  and  later  was 
admitted  to  the  Children’s  Division;  however,  the  staff  had  felt  initially 
that  a correctional  setting  would  be  more  beneficial.  The  problems  in  this 
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case  were  particularly  complex;  the  child,  who  suffered  an  impulse-ridden 
personality,  stubbornly  resisted  ego  and  superego  development. 

The  following  information  was  obtained  about  the  four  children  who 
were  not  referred: 

In  one  case,  the  child  had  already  been  accepted  in  a classroom  for 
emotionally  disturbed  children  at  the  time  the  family  contacted  Fort 
Logan.  The  mother  reported  in  the  follow-up  interview  that  the  child 
responded  well  to  this  placement. 

In  the  second  case,  the  problem  appeared  to  be  only  immaturity  in 
the  child.  The  mother  reported  the  child’s  improvement,  crediting  it  to  her 
self-awareness  of  mistakes  in  her  previous  child-rearing  practices. 

The  third  family  had  contacted  Fort  Logan  pending  the  outcome  of  a 
juvenile  court  hearing.  Follow-up  revealed  that  their  child  had  benefited 
from  placement  in  a correctional  setting. 

In  the  fourth  case,  the  mother  was  uncertain  about  wanting  a 
referral.  We  subsequently  learned  that  the  child  was  not  doing  well,  but 
the  mother  was  still  ambivalent  toward  treatment. 

Outcome  of  Referrals  for  Children  Evaluated  but  Not  Admitted 

Table  2 shows  the  distribution  of  referrals  for  the  42  families  who 
were  evaluated  by  our  staff  but  whose  children  were  not  accepted  for 
admission. 

Again,  a large  percentage  of  families  who  followed  through  with 
referral  reported  benefit.  Circumstances  of  the  unfavorable  outcome  in 
six  cases  who  followed  through  were  as  follows: 

Two  children  eloped  from  correctional  settings.  One  was  believed  to 
be  concealed  by  his  parents.  The  other  was  reported  to  have  left  the 
state  with  some  adults  who  were  trafficking  in  narcotics. 

Two  children  were  in  an  adult  treatment  division  at  the  Colorado 
State  Hospital  because  they  could  not  be  accommodated  in  an  open  setting 
such  as  Fort  Logan. 

One  family  could  not  meet  the  expense  of  a recommended  private 
school  for  perceptually  handicapped  children. 

Ongoing  consultation  for  one  child  had  been  refused  by  a state- 
operated  children’s  home;  the  child  continued  to  adjust  poorly  in  that 
home. 
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TABLE  2 

REFERRAL  DISTRIBUTION  OF  FAMILIES  EVALUATED  BUT 
NOT  ADMITTED 
(N  = 42) 


Disposition 

Outcome 

Satisfactory  Unsatisfactory 

T otal 

Followed  through  with  referral 

To  original  agency 

14 

4 

18 

To  different  agency 

11 

2 

13 

Did  not  follow  through  with 

referral 

7 

4 

11 

Totals 

32 

10 

42 

Among  the  families  who  did  not  follow  through  with  referral,  seven 
children  appeared  to  be  doing  well.  Three  had  improved  following  foster 
home  placements,  and  one  had  improved  following  placement  in  a special 
education  class.  Three  had  shown  spontaneous  improvement. 

Four  children  of  families  who  did  not  follow  through  continued  to 
have  problems.  One  had  come  to  the  attention  of  the  juvenile  court  after 
refusing  referral  to  the  Colorado  Youth  Center.  One  family  had  contacted 
a child  welfare  agency  but  had  not  yet  accepted  foster-home  placement 
for  the  child. 

In  the  third  case,  the  child  was  refused  admission  to  the  state 
training  school.  The  parents  did  not  have  sufficient  funds  for  private 
placement  in  either  a school  for  the  retarded  or  a school  for  the  perceptu- 
ally handicapped. 

In  the  fourth  case,  the  mother  refused  to  allow  the  child  to  be  ad- 
mitted to  the  24-hour  program  in  our  children’s  division.  She  appeared  to 
be  a mother  who  needed  to  maintain  a symbiotic  relationship  with  her 
child. 


DISCUSSION 

Should  we  have  evaluated  the  children  referred  but  not  accepted  for 
evaluation?  Should  we  have  admitted  the  children  who  were  evaluated  but 
not  accepted  for  admission? 
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The  follow-up  survey  indicated  that  for  most  of  the  children  Fort 
Logan  staff  had  made  appropriate  decisions  and  helpful  referrals.  There 
were  almost  no  differences  in  outcome  between  the  families  who  had  only 
telephone  contact  with  our  staff  and  those  who  were  evaluated.  Thirty-one 
families  in  each  group  followed  through  with  referral;  24  (77%)  of  the 
former  group  and  25  (89%)  of  the  latter  reported  favorable  outcomes. 

Twenty-six  of  the  families  referred  did  not  follow  through,  but  13 
of  the  26  reported  that  their  children  had  shown  spontaneous  improve- 
ment, a phenomenon  probably  similar  to  that  often  seen  in  patients 
on  treatment  waiting  lists  of  outpatient  facilities.  In  the  five  other 
reports  of  improvement  in  this  group,  remedial  services  had  been  used  at 
some  point. 

In  addressing  the  concerns  of  the  study,  particular  attention  was 
given  to  the  decisions  made  for  the  children  who  had  not  fared  well. 
Results  were  unsatisfactory  for  22  of  the  92  children.  The  reasons  for 
failure  were  not  always  conclusive,  but  we  noted  some  circumstances 
that  may  have  influenced  outcome. 

In  11  cases,  failure  appeared  to  be  due  primarily  to  minimal  interest 
on  the  part  of  the  family.  Some  of  these  families  made  token  gestures  of 
following  through  with  referral;  others  rejected  the  recommendations 
altogether. 

Only  four  of  the  six  children  referred  to  a correctional  setting 
arrived  there.  Two  children  showed  improvement;  the  other  two  eloped. 
The  staff  of  Fort  Logan  views  the  correctional  agencies  as  treatment 
settings,  but  it  appears  that  not  many  other  agencies  appreciate  their 
treatment  potential.  Present  laws  concerning  the  use  of  correctional 
placements  are  structured  for  repeated  offences,  overlooking  the  pos- 
sibility of  using  such  settings  therapeutically  for  the  early  offender. 

Six  children  appeared  to  be  the  victims  of  service  or  communi- 
cation gaps.  One  agency  had  to  refuse  service  because  it  could  not 
meet  the  child’s  needs,  but  it  failed  to  suggest  other  possibilities  for 
help.  And  in  one  instance,  a state-operated  foster  home  refused  our 
offer  of  psychiatric  consultation  services  while  the  child  remained  in 
the  home.  In  the  other  four  cases,  the  families  had  incomes  too  high 
to  qualify  for  existing  agency  services  but  not  high  enough  to  afford 
private  care. 
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CONCLUSIONS 

The  large  proportion  of  families  who  benefitted  from  referral  back 
to  the  initiating  agency  suggests  that  indiscriminate  admission  of  all 
referrals  to  Fort  Logan  could  bypass  the  professional  skills  still  avail- 
able in  the  referring  agency. 

While  the  study  demonstrated  the  importance  of  careful  screening 
of  applicants  to  avoid  unnecessary  hospitalization,  it  did  not  diminish 
our  concern  about  the  children  who  need  a 24-hour  or  day  care  program 
but  who  are  not  referred  or  who  fail  to  receive  services  because  of  un- 
cooperative families.  Since  most  agencies  are  overwhelmed  trying  to 
help  people  who  want  help,  they  can  scarcely  justify  pursuing  unwilling 
recipients  of  their  care.  One  answer  to  this  problem  might  be  follow- 
up by  a volunteer  group  under  the  auspices  of  a community  professional 
council. 

The  Children’s  Division  will  continue  close  supervision  of  its 
intake  procedures  through  further  studies  and  hopes  to  explore  some 
of  the  problems  revealed  in  this  study  through  its  expanded  role  as  part 
of  the  Southwest  Denver  Comprehensive  Community  Mental  Health  Center. 
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A major  task  of  a child  welfare  worker  in  a protective  services 
agency  is  to  help  the  children  under  his  care  integrate  unhappy  life 
experiences  and  free  their  potential  for  establishing  satisfying  relation- 
ships that  will  ensure  their  further  development.  When  deprivation  and 
inconsistent  parental  care  lead  to  a child’s  separation  from  his  parents, 
it  is  essential  that  the  child  understand  what  is  happening  to  him  and  why. 
This  article  discusses  a specific  activity  that  became  the  medium  through 
which  information  was  conveyed  to  a young  child-preparing  a written  and 
illustrated  account  of  the  child’s  past  and  current  life. 

BACKGROUND  INFORMATION 

Mary’s  parents,  who  were  Sioux  Indians,  had  been  continuously 
involved  in  marital  conflict.  Despite  the  efforts  of  social  workers  for  many 
years  to  provide  rehabilitative  services  to  the  family,  there  was  a pattern 
of  frequent  separations  and  abandonment  of  the  children  by  one  or  both 
parents.  Court  action  resulted  in  awarding  custody  of  the  two  eldest 
children  to  a state  training  school.  Mary’s  mother  placed  the  four  younger 
children  in  boarding  schools  but  kept  Mary  with  her  until  Mary  was  five 
years  of  age,  at  which  time  she  and  her  younger  brother  were  removed  on  a 
court  order  charging  dependency  and  neglect.  Because  the  behavior  of  the 
parents  had  been  the  primary  focus  of  the  workers’  helping  efforts,  they 
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knew  relatively  little  about  the  individual  children  prior  to  their  place- 
ments. When  Mary  was  placed  in  a foster  home,  it  was  explained  to  the 
foster  parents  that  the  caseworker  needed  to  get  to  know  Mary  and  planned 
to  see  her  regularly  outside  the  home  as  well  as  observe  her  within  the 
foster  family  setting. 

During  the  first  weeks  of  foster  care  it  was  evident  that  Mary  did 
not  know  why  she  was  in  the  foster  home.  She  could  not  discuss  her 
feelings  with  the  worker,  but  she  revealed  fantasies  that  her  mother  would 
buy  a house  and  a car  and  then  take  all  her  children  home  to  live  together 
as  a family.  Since  it  was  most  unlikely  that  her  mother  could  improve  her 
level  of  functioning  sufficiently  to  enable  her  to  resume  responsibility  for 
the  care  of  her  children,  Mary’s  fantasy  could  not  be  encouraged.  (5) 
Rather,  she  needed  to  be  prepared  for  continued  foster  care.  Mary’s  ability 
to  cope  with  her  situation  indicated  that  she  could  use  help  in  understand- 
ing and  accepting  what  had  happened  to  her  that  created  the  need  for  a 
substitute  family.  (2)  Although  the  worker  used  play  therapy  on  occasion 
to  convey  ideas  to  Mary,  it  was  the  child’s  interest  in  the  book  department 
of  the  local  drugstore  that  led  the  worker  to  consider  using  the  preparation 
of  a “book”  as  a means  of  interpretation.  On  one  of  their  subsequent  trips 
to  the  drugstore  Mary  was  again  attracted  to  the  children’s  book  section 
and  asked  the  social  worker  to  buy  her  a book.  The  social  worker  re- 
sponded by  offering  an  alternative— that  he  and  Mary  write  their  own  book. 
Mary  was  interested  in  this  suggestion  and  ready  to  talk  about  it.  The  dis- 
cussion led  to  a consideration  of  how  to  make  a book  and  what  materials 
would  be  required. 

The  worker’s  next  trip  with  Mary  was  used  to  select  the  materials 
needed.  Mary  carried  primary  responsibility  for  determining  what  supplies 
were  to  be  purchased  and  she  appeared  to  enjoy  shopping  for  them.  In 
response  to  her  question  about  what  the  book  would  be  about,  the  social 
worker  suggested  that  it  be  a book  about  Mary  herself.  A considerable 
amount  of  time  was  devoted  to  planning  the  book.  It  was  agreed  that 
before  the  next  visit  the  social  worker  would  write  the  story  and  Mary 
would  find  pictures  that  would  illustrate  the  story.  She  selected  pictures 
from  magazines  and  was  assisted  with  her  project  by  the  foster  parents. 

When  the  worker  and  Mary  met  again  the  story  had  been  written  and 
Mary  had  a packet  of  pictures.  Together  they  read  the  story  and  selected 
pictures  to  illustrate  it.  The  story  was  typed  on  large  sheets  of  paper, 
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with  a paragraph  or  two  on  each  page,  allowing  much  room  for  pictures. 
The  process  of  putting  the  book  together  provided  many  opportunities  to 
read  all  or  selected  parts  of  the  story.  Mary  determined  to  a large  extent 
which  parts  needed  to  be  reread  and  discussed.  As  a result,  she  talked 
about  parts  of  her  life  that  she  had  been  unable  to  discuss  earlier  with 
the  worker.  The  following  is  the  narrative  in  the  book. 

THE  STORY  OF  MARY 


In  the  beautiful  land  of  Chief  Red  Cloud  lived  a man  named  John  and  a 
woman  named  Susan.  Red  Cloud  was  one  of  the  early  chiefs  of  the  mighty 
Sioux  Nation.  Red  Cloud’s  people  were  some  of  the  first  people  to  live  on 
the  great  prairies.  They  hunted  wild  animals  to  eat.  The  berries  and  other 
fruits  that  grew  on  the  prairies  and  in  the  Black  Hills  were  good  to  eat. 

John  and  Susan  were  some  of  Red  Cloud’s  people.  They  were  happy 
that  long  ago  he  had  been  their  chief.  Sometimes  John  and  Susan  lived  in 
Red  Cloud’s  land.  This  land  is  called  the  Pine  Ridge  Reservation.  Some- 
times they  lived  in  a city  with  a lot  of  different  kinds  of  people  in  it. 
This  city  is  called  Rapid  City.  John  and  Susan  grew  some  fine  babies. 
Taking  care  of  the  babies  and  finding  enough  food  for  them  was  a hard 
job.  Sometimes  it  was  easier  for  them  to  take  care  of  their  children  and 
get  food  for  them  if  John  would  live  in  the  city  and  Susan  would  live  on 
the  reservation.  Sometimes  John  and  Susan  loved  each  other  very  much. 
Sometimes  they  did  not  love  each  other. 

When  John  and  Susan  had  six  little  children,  Susan  felt  something 
happy  and  wonderful  inside  her  and  knew  she  was  going  to  have  another 
baby.  Susan  took  her  little  children  back  to  the  land  of  Chief  Red  Cloud. 
There  on  the  reservation  she  gave  birth  to  a baby  girl.  Her  name  was 
Mary.  She  was  a pretty  baby  with  beautiful  black  hair  and  sparkling  eyes. 

When  Mary  was  old  enough  to  crawl  and  stand  up  alone,  her  Daddy 
John  was  hurt  badly  in  a car  accident.  He  could  never  walk  again.  He 
could  not  get  food  for  his  family  and  he  could  not  care  for  them.  There 
had  not  been  many  jobs  in  the  city  or  on  the  reservation.  How  sad  he  was 
after  all  this  had  happened. 

By  the  time  Mary  could  walk  well.  Mama  Susan  was  ready  to  have 
another  baby.  Now  Mary  had  a little  brother.  His  eyes  sparkled  too. 

As  the  children  grew  bigger  and  stronger  they  needed  more  food  to 
keep  on  growing.  They  needed  someone  to  help  them  learn  about  new 
things.  They  needed  someone  to  keep  them  from  getting  hurt.  Mama  Susan 
had  to  do  all  this  by  herself.  Even  though  she  loved  her  children,  it  was 
hard  to  care  for  all  of  them  alone. 

In  the  city  there  were  many  helpers.  Some  of  these  helpers  were 
doctors,  some  were  teachers,  and  some  of  the  helpers  were  social  workers. 
The  social  workers  wanted  to  help  Mama  Susan  so  that  she  could  do  the 
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things  she  wanted  to  do  for  the  children  to  help  them  grow  big  and  happy. 
Sometimes  some  of  the  children  had  to  go  away  to  school  or  live  with 
someone  else.  Even  when  there  were  helpers  and  when  some  of  the 
children  were  somewhere  else,  it  was  hard  for  Mama  Susan  to  do  what  she 
wanted  to  do  for  her  children. 

John  and  Susan  talked  to  some  of  the  social  workers  about  what  to 
do  to  help  the  children.  They  decided  that  the  children  would  get  the  best 
kind  of  care  if  they  could  live  with  other  people.  Because  by  this  time 
John  was  very  sick.  It  was  because  John  and  Susan  loved  their  children 
that  they  decided  to  make  these  plans  for  them. 

The  only  way  the  social  workers  could  put  Mary  and  her  little 
brother  in  another  home  was  to  go  to  court.  In  court  the  judge  could  say 
it  would  be  okay  to  find  new  homes.  If  the  judge  would  say  this,  it  would 
mean  that  John  and  Susan  would  be  able  to  let  the  children  go  to  new 
homes. 

Susan,  Mary,  and  her  little  brother  and  the  social  workers  were  in 
court  on  two  days.  They  were  hard  days.  Susan  cried  and  said  she  had 
wanted  to  be  able  to  help  the  children.  She  was  sorry  that  she  had  not 
been  able  to  take  care  of  her  children,  and  she  also  said  she  loved  them. 
Mary  was  unhappy  and  did  not  know  if  she  should  sit  by  Susan  or  sit  by 
the  social  workers.  The  judge  agreed  that  the  best  way  to  help  Mary  was 
to  let  her  live  with  another  family.  Then  she  could  have  all  the  things 
that  Mama  Susan  wanted  for  her.  It  was  not  easy  for  Mary  to  tell  Mama 
Susan  good-bye  because  she  loved  Mama  Susan.  Mama  Susan  was  a good 
person  to  love. 

Mr.  Eikenberry  was  Mary’s  social  worker.  They  did  not  know  each 
other  very  well  before  they  went  to  court.  It  was  hard  for  Mary  to  know 
whether  Mr.  Eikenberry  could  help  her  and  be  her  friend.  Mr.  Eikenberry 
knows  many  children  and  many  families.  One  of  the  families  he  knows 
are  Mom  and  Dad  Thomas. 

Mom  and  Dad  Thomas  have  loved  and  cared  for  many  children.  They 
had  asked  Mr.  Eikenberry  for  another  child  to  love  and  care  for.  Mr. 
Eikenberry  talked  to  them  about  Mary.  They  decided  that  they  wanted  to 
take  care  of  Mary  as  long  as  she  needs  their  home.  Because  Mary  needed 
a home  and  the  Thomases  needed  another  little  girl,  Mr.  Eikenberry 
brought  Mary  to  live  with  the  Thomas  family. 

At  first,  living  with  the  Thomases  was  a little  hard  for  Mary.  They 
did  some  things  differently  from  the  way  Mary  did  them.  Sometimes  Mary 
would  be  terribly  lonely  for  the  people  she  had  known  before. 

The  Thomases  will  be  Mary’s  mom  and  dad  for  as  long  as  she  needs 
them.  They  make  a good  home.  Dad  Thomas  works  in  the  day  so  he  can 
have  food  for  his  family.  Mom  Thomas  spends  her  days  washing  clothes, 
baking  goodies,  and  caring  for  her  children.  Working  and  washing  and 
baking  are  some  of  the  ways  the  Thomases  show  that  they  love  their 
children. 

Mom  and  Dad  Thomas  have  a pretty  little  yellow  house  with  a white 
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fence  around  it.  The  fence  keeps  the  children  from  going  into  the  street. 
Some  of  the  children  need  to  live  with  the  Thomases  a long  time.  Some  of 
the  children  need  to  live  with  the  Thomases  a short  time. 

The  Thomas  family  is  like  an  Easter  basket.  Some  of  the  children 
have  white  skin.  Some  have  brown  skin.  Some  of  them,  like  Mary,  are 
Red  Cloud’s  people  and  have  a nice  tan  skin.  Some  of  the  children  are 
big,  others  are  just  tiny.  A few  are  boys  and  a few  are  girls. 

All  of  them  like  cookies  and  raisins.  They  all  need  to  be  loved  and 
cared  for.  They  all  pray  and  they  all  sleep.  Sometimes  they  cry  and  some- 
times they  giggle.  Not  many  of  them  like  to  go  to  the  doctor  for  a shot. 
They  all  like  to  feel  the  rain  on  their  pretty  faces.  Like  Mary,  they  all 
need  the  Thomases  for  part  of  the  time  they  are  growing  up. 

EVALUATION  OF  THE  TREATMENT  DEVICE 

One  approach  to  evaluating  this  method  of  handling  with  a child 
potentially  explosive  information  about  his  life  is  to  consider  both  the 
potential  benefits  to  him  and  the  difficulties  it  may  create.  In  regard  to 
Mary  it  seemed  that  the  book  had  served  much  of  its  intended  purpose  by 
the  time  it  was  assembled.  The  process  of  planning  the  book,  matching  the 
pictures  to  the  narrative,  and  repeated  discussions  of  the  content  oc- 
cupied weekly  contacts  for  two  months.  Mary  was  then  ready  to  take  the 
book  home  and  give  it  to  the  foster  mother  for  safekeeping.  Mary  chose  to 
store  the  book  on  top  of  the  refrigerator  in  the  kitchen,  where  it  was 
beyond  the  reach  of  the  other  children  in  the  home.  Significance  might  be 
attached  to  her  choice  because  much  of  the  family’s  activities  took  place 
in  the  large  kitchen  of  the  foster  home. 

Mary’s  attitude  toward  the  book  was  one  of  quiet  respect.  After  her 
book  was  completed,  her  attitude  toward  her  early  life  appeared  to  be  one 
of  acceptance.  (3)  The  new  level  of  adjustment  was  effective  to  the  point 
that  it  allowed  Mary  to  move  beyond  expending  all  her  energies  in  trying 
to  deal  with  her  feelings  about  the  past. 

The  foster  parents  were  involved  in  discussions  about  the  book, 
and  they  indicated  respect  for  it  and  its  meaning  to  Mary.  They  knew 
enough  about  Mary’s  background  so  that  they  could  talk  comfortably 
about  the  book  with  her  and  handle  any  questions  she  might  raise. 

The  social  worker  consciously  attempted  to  write  a story  that 
would  be  an  accurate  representation  of  Mary’s  natural  parents  and  their 
strengths  as  parents  as  well  as  their  limitations.  By  discussing  the  story 
in  segments  with  the  child,  the  worker  had  an  opportunity  to  expand  on 
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the  contents.  Talks  about  the  story  as  the  book  was  prepared  were  lengthy 
and  included  many  details  of  Mary’s  background.  The  guideline  in  these 
discussions  was  to  determine  what  the  child  was  asking  for  and  what  she 
was  able  to  use  constructively.  The  theoretical  construct  on  which  the 
social  worker  was  operating  was  that  Mary’s  having  a clearer  understand- 
ing of  why  her  own  parents  could  not  care  for  her  would  make  it  possible 
for  Mary  to  effect  a more  satisfying  adjustment  to  substitute  parents.  (4) 
The  relationship  that  developed  between  Mary  and  the  worker  during  the 
preparation  of  the  book  enabled  him,  in  the  months  that  followed,  to  help 
her  face  new  situations  as  they  occurred,  especially  conflicts  with  other 
children  and  the  death  of  her  father. 

One  danger  in  a social  worker’s  writing  the  story  of  a child’s  life  is 
that  he  may  inadvertently  emphasize  his  wish  to  help  the  child  at  the 
expense  of  giving  an  accurate  account  of  his  actual  life  experiences. 
Safeguards  that  could  be  taken  to  diminish  this  possibility  include  the  use 
of  colleagues,  or  consultation,  for  more  objective  evaluations  as  the  story 
is  being  developed.  Accurate  representation  and  interpretation  must  be  a 
constant  concern  of  the  worker. 

Another  caution  relates  to  a child’s  readiness  and  capacity  to 
integrate  the  information  about  his  life.  There  should  be  a definite  indi- 
cation that  the  child  wants  and  can  participate  in  such  an  experience.  (1) 

Mary’s  immediate  response  to  the  experience  of  writing  the  book  and 
her  later  adjustment  in  foster  care  tends  to  indicate  that  this  treatment 
device  may  have  potential  value  for  other  children  in  similar  situations. 
Writing  a book  about  a child’s  life  experiences  makes  them  graphic  and 
tangible  for  the  child  and  the  worker.  The  book  can  be  used  to  present 
therapeutic  material  of  a sophisticated  nature  on  a level  that  is  meaning- 
ful and  real  to  a very  young  child.  In  Mary’s  case,  the  worker  learned  that 
two  years  later  the  book  was  still  kept  on  top  of  the  refrigerator.  For 
Mary,  who  had  few  mementos  of  childhood,  the  book  seemed  to  serve  the 
purpose  of  being  a part  of  a developing  identity. 
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The  data  available  for  hospital-wide  program  evaluation  in  mental 
health  facilities  usually  come  from  automated  patient  record  systems. 
The  Fort  Logan  Mental  Health  Center  began  collecting  a wide  variety  of 
data  on  its  patient  population  when  the  Center  opened  on  July  1,  1961.  Its 
automated  systemt  contains  demographic,  historical,  psychological,  medi- 
cal, psychiatric,  admission,  and  discharge  information  on  each  patient. 
Automation  has  increased  the  quantity  of  data  that  can  be  retrieved 
quickly  for  administrative,  clinical,  and  research  purposes,  and  the  de- 
mands for  accuracy  within  the  automated  process  have  increased  the 
quality  of  the  data.  However,  in  the  area  of  treatment  evaluation,  most 
automated  systems  still  depend  upon  criteria  such  as  length  of  stay, 
readmission  rates,  and  response  to  treatment. 

None  of  these  traditional  criteria,  either  alone  or  in  combination, 
meet  the  evaluation  needs  of  the  therapeutic  techniques  employed  at  the 
Fort  Logan  Mental  Health  Center.  For  example,  the  relatively  short 
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length  of  stay  for  patients  in  the  Center’s  Alcoholism  Division  does  not 
mean  that  its  program  is  more  effective  than  that  of  the  Adult  Psychiatry 
Division.  Instead,  it  reflects  the  well-defined,  brief  phases  intrinsic  to 
the  Alcoholism  Division  treatment  program  (2).  The  treatment  structure 
at  Fort  Logan  is  based  on  modalities  ranging  from  24-hour  in-hospital 
care  to  outpatient  care  involving  only  a few  hours  a month.  Outpatients 
remain  on  the  hospital  books  every  day  until  discharged.  To  compare  Fort 
Logan’s  length  of  stay  figures  to  those  of  a hospital  whose  population 
consists  entirely  of  24-hour  in-hospital  care  would  grossly  misrepresent 
the  Center’s  philosophy  of  treatment  and  its  ability  to  carry  out  that 
philosophy  (3). 

Applying  readmission  figures  as  an  evaluative  criterion  presents 
certain  problems  for  Fort  Logan.  If  the  readmission  rate  is  defined  as 
the  number  of  patients  who  were  admitted  in  a given  year  who  were  sub- 
sequently readmitted,  Fort  Logan’s  readmission  rate  decreased  from  33% 
to  19.8%  during  the  first  six  years  (9). 

The  treatment  philosophy  of  the  Center  can  alter  the  significance  of 
readmission  figures,  too.  A treatment  team  may  discharge  a patient, 
believing  that  his  experience  in  the  community  will  be  beneficial,  but 
expecting  him  to  return.  In  this  case,  readmission  must  be  viewed  not  as 
a treatment  failure  but  as  a part  of  the  treatment  process  (9). 

Estimates  of  response  to  treatment  frequently  rest  upon  the  clini- 
cian’s rating  of  a patient’s  improvement,  but  does  the  rating  signify  a 
change  in  behavior,  symptom  alleviation,  social  productivity,  or  all  of 
these  and  more  (11)?  Furthermore,  the  usual  rating  of  improvement 
provides  no  baseline  for  judgment. 

Fort  Logan  currently  assesses  response  to  treatment  through  ratings 
in  the  following  areas;  social  and  family  relationships;  self-management; 
social  productivity;  antisocial  behavior;  use  of  leisure  time;  character 
change;  general  symptom  alleviation;  and  self-confidence  and  self- 
reliance.  This  assessment  allows  identification  of  the  area  of  change  and 
individual  clinical  interpretation  of  the  specific  nature  of  that  change. 
However,  these  interpretations  vary  according  to  the  rater’s  discipline, 
experience,  and  knowledge  of  the  patient  (4,  6,  13).  And  there  is  evidence 
that  when  the  person  who  treats  the  patient  also  judges  his  progress,  the 
rater’s  value  system  and  his  attitude  toward  the  patient  affect  this 
rating  (5,  12).  Recognizing  that  traditional  criteria  alone  cannot  provide 
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a complete  picture  of  treatment  effectiveness,  research  staff  members  at 
the  Center  sought  a new  form  of  program  evaluation.  We  envisioned  an 
automated  progress  note  with  two  major  objectives:  “First,  to  obtain  a 
flexible  but  ongoing  picture  of  the  patient’s  goals  as  set  by  the  patient, 
family,  and/or  staff;  second,  to  provide  the  necessary  feedback  essential 
to  research  in  a clinical  setting.”  (12) 

Glueck  (7)  noted  the  numerous  defects  in  the  customary  narrative 
progress  report:  (a)  the  quality  of  reporting  varies  according  to  the  train- 
ing, motivation,  and  workload  of  the  observer,  (b)  the  frequency  of  reports 
tends  to  be  minimal  through  the  absence  of  a convenient  means  for  making 
them,  and  (c)  the  absence  of  a uniform  structure  and  common  vocabulary 
adversely  affects  the  clarity  and  completeness  of  reports.  The  tremendous 
difficulty  in  locating  and  retrieving  specific  information  could  be  added 
to  this  list. 

In  examining  automated  versions  of  records  devised  by  other  insti- 
tutions, several  advantages  were  immediately  evident.  First,  by  providing 
a list  of  statements  and  asking  the  clinical  personnel  to  indicate  which 
were  applicable  to  a given  patient,  the  information  eventually  included  in 
the  chart  would  be  as  comprehensive  as  the  list  of  items  submitted  to  the 
staff  and  would  cover  areas  that  might  be  ignored,  overlooked,  or  for- 
gotten in  the  conventional  hand-written  narrative  form  of  reporting. 
Second,  the  return  of  the  computer-printed  narrative  report  would  serve  as 
a valuable  communication  tool  for  the  teams. 

Although  the  techniques  for  converting  check  lists  to  computerized 
narrative  reports  could  be  adapted  from  existing  systems,  e.g.,  Glueck’s  (7) 
work  at  Hartford  or  Laska’s  (8)  at  Rockland,  the  content  we  desired  was 
not  already  available.  We  believed  that  team  progress  notes  at  Fort  Logan 
should  include  not  only  the  treatment  goals  for  the  patient,  but  also  the 
plans  for  using  community  and  hospital  resources  in  attaining  these 
goals,  and  the  psychological,  behavioral,  and  social  responses  of  the 
patient.  As  far  as  we  knew,  no  one  had  ventured  into  a goal-method- 
response  approach  to  treatment  evaluation  in  an  ongoing  operation  with 
built-in  feedback. 

Faced,  then,  with  the  enormous  task  of  developing  our  own  content 
and  format,  a committee  of  clinical  and  research  personnel  was  formed  in 
1967  to  devise  a pilot  document  for  the  staff  treatment  goal-method  portion 
of  the  proposed  progress  note. 
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Pilot  Work 

Each  team  committee  member  was  asked  to  provide  a list  of  all  the 
goals  of  treatment  their  team  currently  held  for  each  patient.  The  resulting 
list  of  over  600  goal  statements  was  reduced  by  eliminating  duplications, 
combining  and  rewording  goals  of  similar  nature,  and  redefining  those 
whose  meaning  was  unclear  to  committee  members.  The  statements  were 
then  organized  into  the  following  categories: 

1.  Financial  support 

2.  Living  situation 

3.  Medical  goals 

4.  Self-confidence,  self-expression,  and  independence 

5.  Recognition  of  problem  areas 

6.  Symptom  alleviation 

7.  Social  and  interpersonal  relations 

Each  category  listed  a series  of  items  designating  a treatment  goal 
the  staff  might  hold  for  a patient.  Space  was  provided  for  additional 
comments  from  the  staff  rater. 

The  committee  used  the  same  procedure  to  produce  the  lists  of 
treatment  methods.  The  two  parts  were  combined  into  one  form  by  listing 
the  methods  in  each  category  opposite  the  goals  for  that  category. 

To  test  the  form,  we  asked  four  Adult  Psychiatry  Division  treatment 
teams  to  use  it  for  one  month,  completing  each  form  at  the  time  they 
wrote  the  routine  narrative  progress  notes  in  the  patient’s  chart.  We  then 
examined  the  forms  to  determine  (a)  clarity  of  language,  (b)  inclusiveness, 
(c)  difficulties  encountered  in  completion,  and  (d)  general  clinical  re- 
action. 

Clarity  ol  language.  Two  measures  of  clarity  were  used:  verbal 
reports,  and  number  of  redundant  items  added  by  the  raters.  The  teams 
reported  no  difficulty  interpreting  the  meaning  of  individual  items  except 
in  the  category,  “Recognition  of  Problem  Areas.”  It  was  not  clear  to  them 
whether  “recognition”  referred  to  the  team  or  to  the  patient.  The  am- 
biguity was  corrected  by  changing  the  category  heading  to  “Patient 
Recognition  of  Problem  Areas.”  No  redundant  items  were  added  by  the 
raters. 

Inclusiveness.  We  measured  this  aspect  of  the  form  by  the  number 
of  new  items  added  by  the  raters  and  the  distribution  of  checked  items 
over  all  subgroups. 
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In  the  area  of  family  relations,  “husband”  and  “friends”  were 
added  under  the  item,  “Stop  Excessive  Arguing  with.”  In  the  next  form 
revision,  “Stop  Excessive  Arguing  with  Husband,”  was  added  under 
“Marital  Relations,”  and  “Stop  Excessive  Arguing  with  Friends”  was 
added  to  the  area  of  “Peer  and  Employer  Relations.” 

The  distribution  of  items  checked,  by  category,  is  shown  in  Table  1. 
Since  among  all  items  on  the  form,  only  the  various  types  of  bizarre 
sexual  behavior  were  not  checked,  we  condensed  this  subsection  to  a 
single  item  in  subsequent  forms. 


TABLE  1 

PERCENT  OF  ITEMS  CHECKED  BY  CATEGORY 


Goal  Category 

Number  of  checks 

% of  total 

Financial  support 

88 

6.08 

Living  situation 

65 

4.49 

Medical  goals 

Self-confidence,  self,  expression, 

145 

10.03 

and  independence 

489 

33.84 

Symptom  alleviation 

189 

13.07 

Recognition  of  problem  areas 

257 

17.78 

Social  and  interpersonal  relations 

212 

14.67 

Totals 

1445 

99.96 

Difficulties  in  completion.  The  teams  reported  some  difficulty  in 
determining  when  to  check  “Not  yet  a matter  of  concern.”  Additional 
instructions  were  given  that  this  item  was  to  be  checked  when  (a)  there 
was  no  problem  in  the  area,  or  (b)  when  a problem  existed  in  the  area 
but  the  team  wished  to  defer  action. 

General  clinical  reaction.  Verbal  feedback  from  the  teams  in- 
dicated their  favorable  reaction.  They  felt  that  the  goal  list  provided  them 
with  a convenient  way  to  discuss  the  treatment  program  for  a given 
patient.  Having  treatment  goals  spelled  out  and  known  to  all  team  mem- 
bers helped  to  maintain  treatment  unity  for  the  patient. 
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Design  for  Subsequent  Forms  and  Procedure 

The  staff  goal-method  checklist  was  only  one  segment  of  the  total 
progress  note  envisioned.  To  insure  a truly  representative  picture,  we 
needed  the  stated  goals  of  the  patient  and  of  immediate  community  mem- 
bers directly  or  indirectly  involved  in  the  patient’s  hospitalization.  As 
Suchman  (10)  stated,  “There  can  be  little  question  that  values  play  a 
large  role  in  determining  the  objectives  of  public  service  programs  and 
that  any  evaluation  study  of  the  desirable  and  undesirable  consequences 
of  such  programs  must  take  social  values,  especially  conflicting  values, 
into  account.”  Without  goal  statements  from  the  patients  and  community, 
it  would  be  difficult  to  determine  the  relevance  or  “correctness”  of  the 
staff’s  point  of  view  and  the  effect  of  possible  differences  in  perception 
of  goals  upon  the  staff’s  ability  to  treat  patients  (1). 

Thus,  our  immediate  goal  was  to  develop  flexible  instruments  for 
recording  the  treatment  expectations  of  the  patient  and  the  community,  as 
well  as  the  staff,  and  the  level  of  goal  attainment  perceived  from  each 
standpoint. 

From  Fort  Logan  admission  forms  already  on  file,  we  constructed  a 
preliminary  list  of  patient  treatment  goals,  using  the  reasons  patients 
give  for  applying  for  treatment.  Later,  each  patient  will  be  asked  to  in- 
dicate his  goals  for  treatment  either  at  evaluation  for  admission  or  at 
actual  admission.  (Patients  unable  to  provide  goal  data  upon  admission 
may  be  able  to  later  in  treatment.)  The  response  instrument,  the  level  of 
goal  attainment,  will  consist  of  a series  of  statements  describing  the 
progress  the  patient  makes  toward  each  stated  goal.  At  the  time  of  dis- 
charge, the  patient  will  be  asked  to  complete  a second  statement  of  goals 
and  to  rate  his  response  to  both  sets  of  goals. 

Goal  statements  from  community  members  will  be  obtained  either  at 
the  time  of  evaluation  or  admission,  since  relatives  usually  accompany 
the  patient  at  one  of  these  times.  Approximately  one  week  after  the 
patient  is  discharged,  we  will  send  the  response  rating  form  to  the  family 
or  community  members,  asking  that  they  indicate  any  changes  in  or 
additions  to  their  original  goals  and  rate  these  also. 

At  the  time  of  admission  evaluation,  or  within  24  hours  after  ad- 
mission, a treatment  team  member  will  complete  the  staff  goal-method 
checklist.  In  our  current  hospital  procedure,  progress  notes  are  written 
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weekly  for  the  first  month,  then  every  two  weeks  for  two  months,  and  then 
monthly  until  discharge.  Using  this  procedure,  each  progress  note  will 
reflect  the  patient’s  level  of  goal  attainment  for  the  preceding  time  period. 
As  each  note  is  due,  a staff  member  will  complete  another  goal-method 
form,  making  whatever  changes  he  feels  are  needed  in  either  the  goals  for 
the  patient  or  the  treatment  methods  to  be  used. 

The  data  on  all  forms  will  be  transferred  to  IBM  cards  via  optical 
scanner  equipment  at  Fort  Logan,  and  the  cards  will  then  be  processed 
by  computer,  using  NOVEL(8)  computer  language.  Feedback  will  consist  of 

(a)  a print-out  of  the  progress  note  in  narrative  format,  to  be  sent  to  the 
appropriate  staff  member  for  verification,  signature,  and  insertion  into  the 
patient’s  chart;  and  (b)  a separate  list  of  goals  and  response  items,  to  be 
sent  to  the  patient’^  team  for  reference  in  treatment. 

Evaluative  Applications 

The  second  step  in  the  project  will  be  to  assess  (a)  the  direction 
and  degree  of  any  goal  differences  between  staff,  patient,  and  community; 

(b)  the  extent  to  which  these  differences  affect  the  level  of  goal  attain- 
ment; and  (c)  the  extent  to  which  the  rapid  feedback  of  goal  information 
to  clinicians  affects  level  of  goal  attainment. 

The  amount  of  agreement-disagreement  among  staff,  patient,  and 
community  expectations  will  be  determined  by  comparing  the  three  goal 
lists.  The  percentage  of  agreement-disagreement  in  each  category  will  be 
computed.  The  effect  of  each  type  of  disagreement— omission  of  a goal, 
difference  in  degree  of  goal  expectation,  or  difference  in  direction  of 
goal  expectation-upon  the  level  of  attainment  will  be  determined  by 
correlational  techniques.  We  will  test  the  assumption  that  knowledge  of 
the  amount  and  source  of  disagreement  will  lead  to  less  disagreement  and 
will  increase  the  level  of  goal  attainment.  However,  decisions  relevant  to 
reducing  disagreement,  along  with  the  means  for  so  doing,  will  rest  with 
the  parties  involved. 

The  long-range  objective  of  the  research  staff  is  to  obtain  a picture 
of  the  effectiveness  of  Fort  Logan’s  treatment  program  by  combining  the 
goal-method-response  measures  with  traditional  evaluative  criteria  such 
as  length  of  stay  and  readmissions. 

Two  types  of  evaluation  would  be  possible.  The  first  would  be  an 
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ongoing,  weekly  or  monthly,  assessment  of  the  patient’s  progress  toward 
the  predetermined  goals  perceived  by  the  staff.  It  would  be  possible  to 
chart  the  level  of  attainment  for  each  goal  by  area,  i.e.,  living  situation, 
financial  support,  or  symptom  alleviation,  for  any  given  patient  or  group 
of  patients  over  time  or  within  a given  period  of  time.  By  tapping  the 
wealth  of  data  stored  in  the  current  record  system,  we  could  compare  the 
progress  of  various  groups  of  patients  on  such  variables  as  age,  sex, 
diagnosis,  marital  status,  cultural  group,  social  history,  and  mental 
status  factors.  These  comparisons  could  aid  in  determining  which  goals 
are  most  realistic  for  a given  patient. 

The  second  type  of  evaluation  would  measure  the  overall  success  of 
the  treatment  program  based  upon  goal  attainment  pooled  across  all 
patients.  At  the  completion  of  treatment,  we  would  obtain  three  admittedly 
subjective  ratings  of  the  degree  to  which  the  goals  set  forth  were  reached. 
For  each  of  these  viewpoints  an  over-all  score  (a  simple  mean)  could  be 
obtained  by  summing  the  level  of  attainment  (response)  ratings  and  divid- 
ing by  the  number  of  goals  stated.  This  crude  measure  could  then  be 
refined;  for  example,  once  the  data  are  sufficient,  qualifying  factors  such 
as  the  value  placed  on  the  attainment  of  each  goal  could  be  ascertained. 


SUMMARY 

This  report  covers  the  background,  pilot  work,  and  preliminary 
designs  for  the  development  of  an  automated,  goal-oriented  progress  note. 
The  project’s  ultimate  aim  is  a new  dimension  of  hospital  treatment 
evaluation  compatible  with  traditional  assessment  criteria  such  as  length 
of  stay  and  readmissions.  The  planned  goal-method-response  measures 
will  provide  patients,  community  members,  and  hospital  staff  a means  of 
bridging  some  of  the  known  gaps  in  perceptions  of  illness  and  expecta- 
tions of  treatment.  The  checklist  forms  will  be  flexible  enough  to  accom- 
modate changing  circumstances  in  the  course  of  hospitalization,  and  the 
automated  process  will  furnish  rapid  feedback  to  clinicians  through 
computer-printed  narrative  reports. 

The  project  staff  is  currently  engaged  in  further  refining  the  re- 
search instruments  and  testing  the  techniques  for  implementation. 
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CURRENT  RESEARCH* 


Editor’s  Note : This  section  of  the  Journal  briefly  describes  some  of  the 
current  research  projects  at  the  Fort  Logan  Mental  Health  Center.  Addi- 
tional studies  in  progress  will  be  summarized  in  the  Fall-Winter  1970 
issue. 

Adolescent  Division 

Family  Communication,  an  exploration  of  ways  to  improve  family 
communication  and  understanding,  based  on  the  assumption  that  break- 
down in  communication  is  one  of  the  key  etiological  variables  in  families 
producing  adolescent  patients.  The  study  involves  pre-evaluation  of 
family  communication,  the  use  of  structured  audiotape  exchanges  between 
parents  and  child,  and  post-treatment  evaluation  of  family  communication. 
(Principal  Investigator:  Floyd  Martinez,  Ph.D.) 

Stall  Attitudes  Toward  Juvenile  Offenders,  an  investigation  of  the 
congruence  or  divergence  of  attitudes  toward  mental  illness  and  attitudes 
toward  juvenile  offenders.  The  intent  of  the  study  is  to  explore  descrip- 
tively the  attitudes  of  individuals  working  with  juvenile  offenders;  to 
compare  these  attitudes  with  those  of  staff  working  with  the  “mentally 
ill”;  to  help  develop  an  on-going  indication  of  staff  attitudes  for  the 
institution;  and  to  help  develop  an  instrument  to  aid  in  staff  selection 
and  training.  (Principal  Investigator:  Stephen  L.  Bloom,  Ph.D.) 

Alcoholism  Division 

Relating  Outcome  to  Treatment  Methods  and  Pre-treatment  Measures 
for  Alcoholic  Patients,  a study  of  an  experimental  program  begun  January 
15,  1970,  involving  three  treatment  plans:  a standard  treatment  program,  a 
standard  program  with  intense  family  unit  involvement,  and  a program 


*Much  of  the  work  described  here  is  partially  supported  by  General 
Research  Support  Grant  #1-S01-RR05699-01  from  the  National  Institute  of 
Mental  Health. 
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focused  on  community  situations  in  which  an  alcohol  problem  is  involved. 
The  primary  aims  of  the  study  are  (a)  to  relate  treatment  outcome  (four 
months  after  admission)  to  the  three  plans,  and  (b)  to  relate  outcome  to 
alcoholism  dimensions  describing  drinking  patterns  and  socioeconomic 
factors,  measured  on  the  patient  prior  to  treatment.  (Principal  Investi- 
gators: Kenneth  W.  Wanberg,  Th.D.,  and  Donald  M.  Fairchild,  Th.D.) 

Synthesizing  ol  Alcoholism  Patterns,  a study  of  the  interrelation- 
ships of  various  independent  patterns  identified  in  the  excessive  use  of 
alcohol.  The  project  has  two  major  goals:  (a)  to  develop  factor  scores  for 
each  domain  of  data  on  symptom  patterns,  and  (b)  to  develop  standardized 
scores  for  these  factors  to  provide  a normative  basis  upon  which  an 
individual  patient  can  be  scored.  (Principal  Investigator:  Kenneth  W. 
Wanberg,  Th.D.) 

Children’s  Division 

Follow-up  ol  Children  Discharged  from  the  Children’s  Division,  an 
extension  of  the  ongoing  follow-up  of  children  to  include  personal  inter- 
views with  parents  and/or  other  adults  responsible  for  the  children  dis- 
charged, to  elicit  more  specific  information  on  the  current  status  of  the 
presenting  complaints,  and  to  study  school  performance  and  school  pro- 
gress prior  to  hospitalization,  during  hospitalization,  and  after  discharge. 
(Principal  Investigator:  Terry  D.  Keepers,  Ph.D.) 

Treatment  Inquiry  Research  Project,  an  examination  of  the  ways  in 
which  individual  staff  members  perceive  the  problems  of  patients  and  the 
treatment  methods  they  use  in  each  case.  The  aims  of  the  project  are 
(a)  to  identify  the  types  of  problems  we  feel  we  can  treat  successfully 
and  those  we  believe  we  cannot  treat  successfully,  (b)  to  identify  which 
of  a patient’s  problems  we  focus  on  in  treatment,  (c)  to  determine  the 
kinds  of  treatment  individual  staff  members  feel  are  most  appropriate  for 
a given  problem,  (d)  to  measure  the  consistency  or  inconsistency  of 
individual  staff  perceptions  of,  and  efforts  to  treat,  individual  patients 
with  certain  types  of  problems,  and  (e)  to  determine  differences  or  simi- 
larities in  team  treatment  methods  with  patients  who  have  certain  types  of 
problems,  (Principal  Investigator:  Keith  Hammond,  Ph.D.) 
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Crisis  Intervention  Division 

Outcome  of  Crisis  Intervention  Therapy  and  ol  Milieu  Therapy,  a 
study  comparing  the  effects  of  services  provided  by  the  Crisis  Unit  and 
the  Adult  Psychiatry  Division  at  Fort  Logan.  This  work  evolved  from  an 
initial  interest  in  comparing  admission  characteristics  of  patients  ad- 
mitted to  these  services,  with  the  eventual  goal  of  developing  a screening 
instrument  that  would  predict  the  best  treatment  modality  for  each  ad- 
mission. The  expanded  study  considers  the  differences  in  treatment 
approaches  of  the  two  services  as  well  as  the  possible  differences  in 
patient  populations.  The  primary  aims  of  the  project  are  to  compare  the 
effects  of  the  two  treatment  approaches  with  respect  to  (a)  relative  cost, 
(b)  readmission  rates,  (c)  symptom  reduction,  and  (d)  subsequent  com- 
munity adjustment.  Also  to  be  tested  is  the  possibility  that  the  crisis 
intervention  approach  may  favor  patients  with  good  premorbid  social 
adjustment,  whereas  the  milieu  approach  may  favor  patients  with  poor 
premorbid  social  adjustment.  (Principal  Investigators:  Richard  M.  Eisler, 
Ph.D.,  Vail  Williams,  Ph.D.,  and  Rita  B.  Vollman,  Ph.D.) 

Program  Information  and  Analysis  Department 

Development  and  Pretesting  of  a Modified  Progress  Note  for 
Program  Evaluation,  an  adjunct  to  a major  research  project  (described  in 
the  article  beginning  on  page  39  of  this  issue)  to  develop  an  automated 
goal-method-response  patient  progress  note  for  treatment  program  evalua- 
tion. The  study  design  proposes  major  modifications  in  the  current 
progress  note  that  will  (a)  simplify  the  process  of  testing  and  refining  the 
current  note,  (b)  identify  the  parts  of  it  that  are  introducing  large  amounts 
of  error  into  the  data,  (c)  identify  sources  of  individual  differences  among 
staff  in  setting  goals,  (d)  increase  the  power  and  usefulness  of  the  data 
ultimately  collected,  and  (e)  provide  comparisons  between  the  existing 
progress  note  and  the  proposed  revised  form  and  between  the  revised  form 
and  variables  such  as  length  of  stay,  social  history  information,  response 
to  treatment  items  on  the  discharge  summary,  and  perhaps  others.  (Princi- 
pal Investigators:  Thomas  H.  Smith  and  Isabel  Cinnamon) 

Cost-Benefit  Analysis  of  Mental  Health  Programs,  a proposal  to 
study  existing  Fort  Logan  data  to  develop  preliminary  cost-benefit 
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estimates  and  to  develop  a weighting  method  based  on  utility  theory  to 
apply  economic  values  to  intangible  benefits.  (Principal  Investigator: 
Paul  R.  Binner,  Ph.D.) 

Pre-  and  Post-hospitalization  Community  Adjustment,  an  ongoing 
follow-up  study  of  discharged  patients  that  focuses  on  four  major  areas  of 
adjustment:  family  and  social  relationships,  social  productivity,  self- 
maintenance, and  antisocial  behavior.  The  data  have  been  examined 
tentatively  from  a number  of  aspects;  the  current  focus^  is  on  the  attrition 
rate  over  time,  comparing  the  characteristics  of  “lost”  patients  with 
those  of  patients  about  whom  information  is  still  collected.  (Principal 
Investigator:  Hiram  Gordon,  Ph.D.) 

Staff  Development  Department 

Three  evaluation  projects  in  this  department  are  partially  supported 
by  U.S.P.H.S.  Grants  2-T01-MH10871-04,  5-T01-MH10412-05,  and  5-T01- 
MH08723-07  from  the  National  Institute  of  Mental  Health: 

Psychiatric  Residency  Evaluation,  involving  the  development  of  a 
rating  form  for  supervisory  assessment  of  resident  progress  in  training. 
The  project  includes  an  intensive  study  of  six  residents  during  the  first 
nine-month  residency  placement,  focusing  on  resident,  supervisor,  and 
head  nurse  expectancies  for  the  resident.  (Principal  Investigator:  Samuel 
B.  Schiff,  M.D.) 

Psychology  Internship  Evaluation,  involving  the  development  of  a 
rating  form  for  supervisory  assessment  of  intern  progress  while  in  train- 
ing. The  project  includes  a pre-  and  post-investigation  of  how  treatment 
ideology  changes  during  the  year  of  internship  at  the  Fort  Logan  Mental 
Health  Center.  (Principal  Investigator:  Nuri  A.  Assafi,  Ph.D.) 

Staff  Development  Study,  focusing  on  increased  sensitivity  of 
treatment  programs  to  the  needs  of  the  poor  from  the  black  and  brown 
minority  groups.  Evaluation  consists  of  helping  staff  to  state  outcome 
objectives,  thus  providing  criteria  against  which  to  measure  the  effective- 
ness of  treatment  efforts.  (Principal  Investigator:  Samuel  B.  Schiff,  M.D.) 


The  Fort  Logan  Mental  Health  Center  is  Colorado’s  second 
state  hospital.  Currently  serving  almost  half  the  population  of  the 
state,  its  organization  follows  as  much  as  possible  the  recommen- 
dations of  the  Joint  Commission  on  Mental  Illness  and  Health. 
Concepts  of  milieu  therapy  are  strongly  utilized,  with  emphasis 
on  expansion  of  professional  roles  and  the  involvement  of  the 
patient’s  family  and  his  community  in  treatment.  The  hospital  is 
entirely  open  and  relies  heavily  on  transitional  forms  of  treatment. 
Approximately  one-half  of  its  patients  are  admitted  directly  to  day 
care,  and  evening  care  is  offered.  Geographic  and  administrative 
decentralization  are  utilized,  with  the  same  psychiatric  team 
following  the  patient  from  the  time  of  admission  through  all  phases 
of  treatment. 
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